Four of Mathematics 
Assessment (terns 



JANE GREENLEES 

makes reference 
to four comic book 
characters to make 
the point that together 
they are a formidable 
team, but on their own 
they are vulnerable. 
Read on to find out 
the Fantastic Four of 
Assessment and how 
they work together. 


S tandardised tests are becoming an ever 
increasingly important presence in 
schools today. In fact, national meas¬ 
ures of academic performance have the 
potential to be the driving force behind 
teaching practice (Lowrie & Diezmann, 2009; 
Pedulla, Abrams, Madaus, Russell, Ramos & 
Miao, 2003). In 2008, the introduction of the 
National Assessment Program (NAPLAN) in 
Australia increased the emphasis on national 
standards and benchmarks and created greater 
forms of accountability. Student performance 
became a reflection of the teacher, staff, 
principal, school and State. With such high 
stakes involved, it can be anticipated—based 
on international evidence—that teachers 
will abandon personal teaching beliefs and 
focus on “teaching to the test”. However 
the Ministerial Council for Education, Early 
Childhood Development and Youth Affairs 
(MCEECDYA) argues that success is found 
in teaching numeracy basics as “the tests 
reflect core elements of state and territory 
curricula”. 

What it fails to acknowledge is that the 
entire nature of test design has changed 
dramatically in recent years and has moved 
beyond simply mathematical content. As 
Lowrie and Diezmann (2009) maintain, 
“substantial data is obtained (and reported) 
on student performance on mathematics, 
but rarely do we consider whether the 
tasks actually assess student knowledge and 
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numeracy understandings” (p. 8). In this 
paper, I examine the four components of 
assessment items and the need for implicit 
instruction within the classroom for student 
success. Just like the “Fantastic Four” of 
Marvel comics fame, each component has 
its own unique ‘power’ that makes them 
strong enough to stand alone but as a group 
can be a force to be reckoned with. These 


are mathematical content, literacy demand, 
contextual understanding, and graphics (see 
Figure 1). 



Figure 1. The fantastic four of mathematics assessment items 
(MYCEETYA, 2009a: Year 5 Numeracy Test, Item 24). 


1. Mathematical content 

Mathematical content is at the heart of all 
assessment, examining what the students know. 
They are the fundamental basics children are 
taught throughout their school career and 
are often the standards used by parents to 
ascertain how well their child is doing. While a 
national curriculum is still being developed in 
Australia, its foundation has been laid in the 
release of the National Statements of Learning 
far Mathematics (NSL) (MCEETYA, 2006) in 
conjunction with the NAPLAN tests. While 
most curriculum documents of Australian 
education jurisdictions are organised into 
stages or levels, the NSLs focus on the four 
years used within the national assessment. 
They are Years 3, 5, 7 and 9. Subsequently 
these year levels are structured around “five 
broadly defined and inter-related aspects of 
Mathematics curriculums that are considered 
essential and common” (MCEETYA, 2006, p. 3). 


These are: 

• Working mathematically — problem 
solving, representation and modelling, 
investigating, conjecturing, reasoning 
and proof, estimating and checking the 
reasonableness of results or outcomes (see 
Figure 2); 

• Number — representation and models for 
number, counting, magnitude, order and 
exact or approximate calculations (see 
Figure 3); 

• Algebra, function and pattern — working 
with functions and relations applied to 
everyday and mathematical objects, 
patterns in number and space, and 
general forms (rules, formula, tables, 
graphs, equations and equivalences) 
expressed using words, symbols or 
diagrams (see Figure 4); 

• Measurement, chance and data — 
length, area, volume, angle, mass, time, 
temperature, probability and statistics (see 
Figure 5); 

• Space — identification, classification 
and representation of 2D and 3D shapes 
and objects. 



Figure 2 .Working mathematically (MYCEETYA, 2009a: Year 3 
Numeracy Test, Item 1). 


Each beetle has 6 spots. 


Which of these shows one way to work out the total number of spots? 


3 + 3 + 3 


6 + 3 


6-3 


6 + 6 + 6 

V ° 

o 
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Figure 3. Number (MYCEETYA, 2009a: Year 3 Numeracy Test, Item 9). 
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The fantastic four of mathematics assessment items 



Figure 4. Algebra, function and pattern (MYCEETYA, 2009a: Year 3 
Numeracy Test, Item 2). 



Altogether, how many mats will cover the whole Ooor? 

_ J 


Figure 5. Measurement, chance and data (MYCEETYA, 2009a: Year 3 
Numeracy Test, Item 20). 


It is generally assumed that effective 
teaching of mathematical basics would be 
reflected in student success in standardised 
tests. However by failing to acknowledge 
other components of an assessment item 
and the need for explicit instruction in 
the classroom, students are disadvantaged 
and subsequently results affected. As Perso 
(2009) notes, “one particular concern that 
has been raised in relation to the items on 
the tests relates to the literary demands 
inherent in understanding the requirements 
of the tasks” (p. 14). Therefore a student may 
know the mathematics but lack the ability to 
coherently read the question. 


conventions. These include developing 
meaningful and applicable definitions of 
mathematical terminology, identifying words 
that may have one meaning in normal 
discourse and a different meaning in the 
context of a mathematics problem, and the 
use of homophones and similar sounding 
words. For example, in the 2009 Year 3 
NAPLAN the word “flipped” was used as 
shown in Figure 6. It has been argued that 
this term has an everyday meaning and a 
different mathematical meaning that students 
need to identify appropriately (Lowrie & 
Diezmann, 2009). Therefore children who 
have never been exposed to this term in a 
mathematical sense would be disadvantaged 
in such an assessment. 

As such, learning scenarios need to be 
provided for students to practise and become 
familiar with mathematical jargon and 
concepts. This includes utilising strategies 
similar to literacy tools; e.g., building 
vocabulary through the use of dictionaries, 
thesauruses, and discussing, sharing and 
tabulating meanings of words (Perso, 2009). 
Teachers also need to provide opportunities 
for students to unpack the vocabulary 
embedded in mathematical tasks through 
specific processes associated with “working 
mathematically” including questioning, 
communicating and reasoning. 



Which shows the map of Australia flipped over the dotted line? 


oo 


o 
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2. Literacy demand 
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When reading the words in mathematics, 


o 


O 
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just as within any language, children need Figure 6 Mu | tip | e meanings (MYCEETYA, 2009a: Year 3 


to contend with and understand its many Numeracy test, Item 26). 
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This is a train timetable. 


DEPARTURE TIMES 

Cenlral 

6:20am 

9:50 am 

2:52 pm 

7:00 pm 

Rainer 

6:31am 

10:01 am 

3:03 pm 

7:11pm 

Bradford 

6:43 am 

10:13am 

3:15pm 

7:23 pm 

St Marks 

6:53am 

10:23 am 

3:25 pm 

7:33 pm 


What time does the last morning train depart from Central? 


_ J 

Figure 7. The role of context (MYCEETYA, 2009, Year 5 Numeracy test, 

Item 14). 

Another problem is the superficial attention 
children give to the text within a task by 
simply focusing and finding relevant keywords 
(Lowrie & Diezmann, 2009). As such, “this 
can hinder students’ holistic understanding 
of the task, and hence, the rationality and 
correctness of their answers” (Weist, 2003, 
cited in Lowrie & Diezmann, 2009, p. 9). 
It is therefore important to reinforce the need 
for students to read thoroughly and carefully 
through questions and relevant information, 
as well as utilising step-by-step strategies before 
attempting to solve them—in other words, less 
rush more fuss. 

Being numerate, therefore, is more than 
being able to perform the mathematics: it 
also requires a certain degree of literacy skill. 
While many teachers focus on thinking and 
working mathematically in the classroom, 
students also need to be taught how to “read 
mathematically” if they are to be successful 
in the current assessment environment. In a 
similar way, teachers also need to acknowledge 
the role the context of the question plays in a 
student’s understanding of an item. 

3. Contextual understanding 

The NSL outline in the Year 3 Working 
Mathematically strand states that students 
will “actively investigate everyday situations 
as they identify and explore mathematics” 
(MCEETYA, 2006, p. 5). The concept of 
making mathematics ‘real’ by incorporating 


everyday contexts has been a growing 
trend in schools for the past 30 years 
(Boaler, 1994). It is believed that such an 
approach would help students realise the 
relevance of mathematics as it is applied 
to their world outside the classroom. 
Consequently, mathematics is no longer 
presented as a dry, abstract subject of 
‘absolute truths’ but rather one which 
encourages flexibility and creativity thus 
enhancing motivation, confidence and 
enjoyment (Walkerdine, 1989, cited in 
Boaler, 1994). 

As a result, test designers are attempting 
to make questions more realistic and possibly 
authentic. For example, the 2010 Year 5 
NAPLAN included a question where children 
needed to calculate the date of the next 
meeting if they are held on the first Tuesday 
of each month and the last meeting was on 
the 6 March. No calendar was used within 
the question; it only contained the written 
instructions. As such, students needed to have 
an understanding of the formation of weeks, 
months and calendar conventions in order 
to draw the right conclusion. However it is 
believed that this can be problematic for some 
students, particularly low performing students 
who draw upon their everyday knowledge 
that is not specifically relevant to the task. On 
this occasion, many students overly relied on 
their understanding of the number of days in 
a month, identifying March as having 31 and 
April having 30, and subsequently made the 
meeting date 5 April, one day earlier than the 
month before. This example highlights the 
need for various contexts to be utilised in a 
classroom so students can practise applying 
appropriate strategies and understandings 
rather than resorting to routine and familiar 
concepts. This goes beyond teaching basic 
mathematical understanding to applying it to 
real-life situations. 

However it is also argued that mathematics in 
context often requires the student to “suspend 
reality and ignore their common-sense in 
order to get a correct answer” (Boaler, 1993, p. 
556). In the 2007 NAPLAN practice questions, 
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an atypical graphic (Diezmann, 2008) was 
included, with a grid being overlaid on a map 
of Australia to identify large areas. However, in 
the ‘real’ world, grids are more typically used 
on regions with similar characteristics (e.g., 
geographic terrain, States or Territories). The 
result of this kind of situation is categorised 
as “context conditioning” (Boaler, 1994). 
Therefore student performance is often a 
result of how well they recall a method that 
last proved successful in a similar question, not 
necessarily how well they apply mathematical 
understanding. Teachers need to acknowledge 
that while the use of contexts can motivate 
students, they will only enhance learning 
transfer if they are able to offer a “realistic 
and holistic view of mathematics which makes 
sense to students both in the classroom and in 
the real world” (Boaler, 1994, p. 559). 

What also needs to be recognised is the 
important role individuals play in creating 
meaning and imagery as they “adopt, 
develop and invent sign-uses in the contexts 
of teaching, learning, doing and reflecting 
on mathematics” (Ernest, 1997, p. 1). This 
is particularly relevant when considering 
mathematical assessment, as it is often assumed 
that all children will have a similar knowledge 
and understanding of mathematical signs, 
symbols and language. 

The symbols and language used to describe 
mathematics concepts are certainly not static 
and the meaning created and constructed 
from one word or symbol can be strongly 
influenced by who the child is. Yet children 
often have little chance to represent these 
possibilities in a test situation and are simply 
given the option of a right or wrong response. 
Every child is unique and it therefore cannot 
be assumed that in mathematics there is a 
generic code by which all students will abide. 
For example, a child in a metropolitan city 
and a child in a small rural town are exposed 
to vastly different cultures. Therefore the 
inclusion of a train timetable in the 2009 Year 
5 NAPLAN (see Figure 7) could be viewed as 
advantaging one group over another. For one 
child, the item may seem foreign, included 


only in textbooks, while for another it is a part 
of their daily lives. Contexts cannot offer a 
unique meaning to all students and therefore 
when used within an assessment must 
necessitate recognition that different students 
are likely to interpret the same mathematical 
situation in many different ways. 

Howeverwhat also needs to be acknowledged 
is the increasing number of graphical and 
visual representations embedded within 
assessment items and the impact on student 
performance. 

4. Graphics 

Graphics can be categorised as graphs, maps, 
diagrams, charts and tables (Diezmann & 
Lowrie, 2007). Within this technological age 
they have become increasingly important 
in representing and organising information 
(Harris, 1996, cited in Lowrie & Diezmann, 
2007). There have been a growing number 
of graphical and visual representations 
embedded within assessment items. An 
analysis of the 2009 Australian National 
Mathematics Assessment revealed that 23 
out of 35 (66%) questions in the Year 3 
and 26 out of 40 (65%) in the Year 5 test 
contained a form of graphic. With such 
a high prevalence in national tests, there 
needs to be more explicit teaching practices 
in reading, producing, understanding and 
decoding graphical representations. This 
includes focusing and teaching the three 
levels of graphical comprehension as noted 
by Curcio (1987): 

1. Reading the data — requires the literal 
reading of the graph. 

2. Reading between the data — includes 
the interpretation and integration of the 
data in the graph, e.g., the ability to 
compare quantities. This is the level of 
comprehension most often assessed on 
national tests. 

3. Reading beyond the data — requires 
the reader to predict or infer from the 
data information that is neither explicitly 
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nor implicitly stated in the graph. These 
questions seem to be the most challenging; 
examples are hard to find and therefore 
are not utilised in the classroom. 

For many primary-aged students, the 
comprehension of the graphic may be the 
most demanding aspect of a mathematics 
task. If the student is unable to access and 
interpret the information effectively, the 
actual mathematics involved is irrelevant. 
Thus, “students’ performance may be a 
measure of their ability to comprehend the 
graphical (or linguistic) components of a 
task rather than a student’s knowledge of 
the mathematics within the task” (Lowrie & 
Diezmann, 2009, p. 8). 

Friel, Curcio and Bright (2001) suggest 
a guideline for creating a progression 
of sequential development of graph 
comprehension in K-8 grade levels: 

• Grades K-2: object graphs, picture graphs, 
line plots, bar graphs (with use of grid 
lines to facilitate reading; labeling of bars 
with numbers); 

• Grades 3-5: bar graphs (stacked or using 
multiple sets of data), stem plots, pie 
graphs (reading primary emphasis); 

• Grades 6-8: pie graphs (reading and 
constructing), histograms, box plots, line 
graphs. 

Friel, Curcio and Bright (2001) recommend 
that developing mathematics knowledge 
should progress along a continuum K-8, as 
well as the complexity of the data. They also 
note that in order for students to gain a deep 
knowledge of graphs and to make and use 
graphs effectively, they need instructional 
materials that are carefully constructed. This 
includes providing opportunities for students 
to read beyond the data by creating their own 
graphs, working backwards and gaining a 
true appreciation of the structure and design 
of the graphic. 


Conclusion 

In today’s technological age, information is 
represented in visually rich modes, with quite 
different language, context and graphics 
demands. Multiple representations in graphs 
are regularly found in computer games 
and in the media. It is not surprising that 
recently introduced national instruments 
have followed a similar theme. Therefore 
if students are to be successful they need 
to be explicitly taught all aspects of these 
representations since the interrelated nature 
of the design components can be complex “in 
their own right”. This includes incorporating 
all aspects of the curriculum in teaching, 
drawing attention to the use of mathematical 
terminology, exploring the complexity of 
graphics and incorporating different contexts 
within the classroom. 

Similarly, test designers need to pay 
particular attention to making questions 
accessible, clear and fair. With the federal 
government seemingly determined to 
extensively publishing student results, more 
research needs to be conducted to examine 
the design of mathematics items and its 
impact on student performance. If testing 
is going to be given such a high priority, we 
need to closely examine what we actually are 
assessing and whether it is a true indication 
of students’ mathematical knowledge. 
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